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Students’ Perceptions of the Effectiveness of Assessment Feedback as a 
Learning Tool in an Introductory Problem-solving Course 


Abstract 

There have been calls in the literature for reforms to assessment to enhance student learning (Shepard, 2000). 
In many instances, this refers to the need to move from traditional assessment procedures that are 
characterized as content-heavy, summative, and norm-referenced approaches to more constructivist and 
student-centred approaches, often characterized as more “...flexible, integrative, contextualized, process 
oriented, criteria referenced and formative” (Ellery, 2008, p. 421). Whereas summative assessment techniques 
rarely allow students to act on the feedback provided, formative feedback provided throughout the learning 
process can be used to improve future work and promote learning (Ellery, 2008; Higgins, Hartley & Skelton, 
2002) by providing students an opportunity to learn from mistakes. 

Allowing students to learn from their mistakes makes good pedagogical sense. To date there has been little 
research examining students’ use of feedback (Higgins, Hartley, and Skelton, 2002). In an effort to begin to 
add to the literature in this area, this paper describes a study that explored the effectiveness of oral and written 
formative feedback when students were provided the opportunity to use it. The paper begins by reviewing 
literature related to assessment and how assessment relates to feedback in general. It then presents what the 
research has found in relation to students’ perspectives of effective feedback and how they use it. The paper 
ends by presenting the results and discussion. 

La documentation fait etat de demandes de reforme de 1’evaluation pour ameliorer l’apprentissage des 
etudiants (Shepard, 2000). Dans plusieurs cas, cela traduit le besoin de passer des procedures devaluation 
traditionnelles caracterisees par la lourdeur de leur contenu, par leur aspect sommatif et par leurs approches 
normatives a des approches plus constructivistes et centrees sur les etudiants, souvent qualifiees de plus « ... 
souples, integratives, contextualisees, axees sur les processus, balisees par des criteres et formatives » (Ellery, 
2008, p. 421). Alors que les techniques devaluation sommative permettent rarement aux etudiants de se 
conformer a la retroaction fournie, la retroaction formative tout au long du processus d’apprentissage peut etre 
utilisee pour ameliorer les travaux futurs et favoriser l’apprentissage (Ellery, 2008; Higgins, Hartley et Skelton, 
2002) en donnant l’occasion aux etudiants d’apprendre de leurs erreurs. 

Sur le plan pedagogique, permettre aux etudiants d’apprendre de leurs erreurs a du sens. A ce jour, il y a eu peu 
de recherche sur l’utilisation que font les etudiants de la retroaction (Higgins, Hartley et Skelton, 2002). Le 
present article se veut un ajout a la documentation dans ce domaine. Ses auteurs decrivent une etude qui porte 
sur l’efhcacite de la retroaction formative orale et ecrite lorsque les etudiants ont eu l’occasion de l’utiliser 
subsequemment. Les auteurs commencent par une analyse de la documentation sur revaluation et sur les liens 
generaux entre cette derniere et la retroaction. Ils presentent ensuite les resultats de recherche liee aux 
perspectives des etudiants sur la retroaction efficace et sur l’utilisation qu’ils en font. Enfin, ils terminent par 
une presentation des resultats et par une discussion. 
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Sadler (2005) summarized the arguments found in the literature for using criteria-based 
assessments. One of Sadler’s arguments for the use of criteria-based feedback is based on 
fairness: “students deserve to be graded on the basis of the quality of their work alone, ... and 
without regard to each student’s previous level of performance” and “students deserve to know 
the criteria by which judgements will be made about the quality of their work” (p. 178). 

Sadler (2005) also noted a wide range of practice with respect to the use of criteria-based 
assessments in both theory and practice at the post-secondary level. Based on this review of the 
literature, he supported the practice of “specifying qualitative criteria for student responses...” 
(p. 185) because there is transparency in the grading process when reviewing (grading) student’s 
work and such practice provides them with a resource to form and evaluate their work. 

Although for many students and instructors, assessment is synonymous with grading, it is 
in fact a complex and multi-faceted practice (Fenwick & Parsons, 2009). Alverno College 
Faculty (1994) present a helpful framework for student assessment which breaks the process 
down into the following components: 1) defining explicitly observable criteria for effective 
performance (this involves both a description of what behaviours are sought and of the expected 
level of quality with which they might be demonstrated in a particular context); 2) creating a 
stimulus that elicits a performance (e.g., a task or assignment); 3) observing that performance to 
characterize it in terms of the criteria; 4) judging the quality of the performance relative to the 
criteria; 5) sharing results of that judgment with the person performing, using the language of the 
criteria; and 6) providing advice to help the person improve subsequent performances. The 
suggestions of Sadler (2005), Ellery (2008), and Higgins, Hartley, and Skelton (2002) fit well 
within Alverno College’s framework. 

A key component of student-centred assessment is feedback, but in their review of the 
literature, Black and Wiliam (1998) point out that various definitions of “feedback” exist. Black 
and Wiliam’s (1998) definition is "...any information that is provided to the performer of any 
action about that performance" (p. 37). Further, the source can be internal or external, it can be 
compared against a reference standard or can be evaluated on its own terms (not against an 
explicit standard). Ramprasad (1983) defined feedback for use in management theory as 
information that is used to close the gap between actual performance and the reference level. 
According to Ramprasad, if the information is not used to alter performance it should be labelled 
as information or variance, rather than feedback. Applying Ramprasad’s work to education, 
Ellery (2008) stated that, “...real learning takes place when feedback is used in ways that help 
close the gap between where students are (‘actual level’) and where they need to be (‘reference 
level’)” (p. 422). According to Ellery, the process of establishing the size of the gap between the 
actual and reference performance level is through assessment. Assessment generates both a 
judgment about the gap and formative feedback about the elements of performance that need to 
be changed to close it. These activities are represented in steps five and six of the general 
assessment framework outlined above (Alverno College Faculty, 1994). 

Higgins, Hartley and Skelton (2001) have stated that “feedback does not realize its full 
potential to become an integral part of the learning process” (p. 207) and suggested the reason 
for this may be a result of poor communication between the instructor and the student. To 
overcome this problem, they put forth the notion of “feeding forward” information to students 
prior to beginning an assignment rather than “feeding back” after the assignment has been 
completed. Providing students with the criteria upon which they will be assessed before they 
actually begin writing the assignment is an example of feeding forward. This work is consistent 
with Ellery (2008), since it focuses on closing the gap between actual and desired student 
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performance. It also embodies Step 1 of the Alverno College framework that focuses on defining 
explicitly observable criteria for effective performance. 

What is Effective Feedback? 

From a student’s perspective, feedback is one of the most useful products of assessment. 
Gibbs and Simpson (2003) argue that assessment supports student learning when: (a) it sets high 
standards; (b) causes students to engage in sustained and productive work; (c) arrives in time to 
be used to guide further study; (d) is intelligible to students, and (e) focuses on the work rather 
than the students themselves. Higgins, Hartley, and Skelton (2002) explored university students’ 
understanding of instructor feedback. Students indicated the importance of timely, specific, and 
clear feedback from instructors. Comments that directly related to the assignment were found to 
be most helpful as opposed to general feedback provided to the entire class. Other research that 
has explored students’ feedback preferences has found that students preferred feedback 
comments that 

• were directed toward the task as opposed to the self (Black & Wiliam, 1998; Orsmond, 
Merry, & Reiling, 2005); 

• addressed both the strengths and weaknesses of the work, that explained mistakes, and 
provided advice about how to improve their arguments (Black & Wiliam, 1998; Lizzo & 
Wilson, 2008); 

• provided advice on how to improve their critical analysis (Lizzo & Wilson, 2008); 

• helped them “focus on skills relating to a deep approach to learning” (Higgins et al., 
2002, p. 61; McDowell, Smailes, Sambell, Sambell & Wakeline, 2007) and 

• would help them on future assignments, and was fair (described as student friendly, 
legible, consistent and non-contradictory) (Lizzo & Wilson, 2008). 

How Students Use Feedback 

Considering that students at the tertiary level read and pay attention to instructor 
comments it is clear that the provision of feedback is necessary for student learning (Higgins et 
al., 2002). Orsmond et al. (2005) found students use feedback in various ways: (a) as motivation 
to work harder in order to improve understanding of the content thus increasing the likelihood of 
achieving better grades; (b) to talk with the instructor about their work; (c) increase learning 
about how to improve future work; (d) to reflect on their work and, in some cases, approach it 
from a different perspective; (e) to clarify their understanding of their performance as it relates to 
the grades they received, and; (f) to focus on instructors’/assignment expectations, especially 
when feedback is provided before a subsequent assignment. 
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Assessment for Learning 

Literature in the field of assessment has identified characteristics of effective feedback 
and preferred feedback (from the students’ perspective). Recent research (Gibbs & Simpson, 
2004; Hattie & Timperley, 2007; Price, Handley, Millar, & O’Donovan, 2010) has commented 
on the varying roles of assessment and feedback, noting, for example, that assessment is too 
often used as a measurement tool or for grade justification rather than for learning purposes. 
Such notions are directly tied to Ramprasad's (1983) definition of feedback being used to close a 
performance gap, Ellery’s (2008) notion of real learning, and Hattie and Timperley’s (2007) 
notions of feed forward information (feedback provided must address future work). 

Price, Handley, Millar, and O’Donavan (2010) noted the difficulty in attempting to 
measure the effectiveness of feedback. They began to explore students’ engagement with 
feedback by interviewing a select group of 15 university students and 20 university professors. 
One of the purposes of their work was to explore the students’ and professors perspectives on the 
effectiveness of the feedback they received/provided. Their findings echoed those found in 
earlier studies (e.g. be truly effective and to be considered feedback, students must have the 
opportunity to use the feedback to improve future performance) with the addition that students 
and professors differed in their perceptions of the use and quality of feedback and the students’ 
desire for dialogue about the feedback. Biggs (1999) indicated that the combination of multi¬ 
channel feedback referenced to consistent assessment rubrics that are made available to students 
and repeated problems works synergistically to get students to focus attention on where they 
need to grow. 

The literature has identified both characteristics of effective feedback and those that 
students say they prefer. Consistent with this body of research, and taking into account the idea 
that students must be provided opportunities to use the feedback provided, a course was 
intentionally designed that implemented characteristics the literature identified as contributing to 
student learning. More specifically, the feedback practices intentionally implemented included: 
(a) consistent and transparent assessment via the use of a rubric that contained explicit criteria 
directly related to grades; (b) the opportunity for students to use feedback provided on 
subsequent work; and (c) the use of multi-channel feedback (written, verbal, individual and 
group). This study set out to explore students’ perspectives of the effectiveness of these three 
assessment practices. 

Course Description 

Students in this study were enrolled in the course Mathematical and Economic 
Approaches to Problem-Solving, a required component of the Bachelor of Philosophy in 
Interdisciplinary Leadership Studies offered at the University of New Brunswick in Canada. For 
more detailed information on the context and structure of the program in which this course was 
embedded, see Zundel and Mengel (2007). This particular course required students to solve four 
complex math problems using a structured problem-solving approach and economic reasoning. 
Each of the problems required students to build mathematical models on Microsoft Excel 
software to organize the quantitative information about the problem and compare the 
attractiveness of at least three solutions that could potentially meet the objectives. With the 
written description of the problem, students received an assessment rubric (see Appendix A) that 
contained a list of observable criteria such as “Identify constraints, objectives and resources.” 
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These criteria were built on a conceptual model of structured problem-solving (Renaissance 
College, 2007) being introduced to the students for the first time in their program. 

The course design was informed by elaboration theory (Reigeleuth & Stein, 1983). That 
is, the course involved engaging students in a series of similar activities of growing richness and 
complexity. The first of these is the “epitome,” representing the most basic form of the problem 
or activity that contains all the elements of the ability being developed. In subsequent attempts, 
students are given richer, more comprehensive or challenging examples of the same basic 
activity and “elaborate” their ability to work with it (Reigeleuth & Stein, 1983). 

Although the nature of the problems changed substantially over the four attempts (see 
Table 1 for a description of the relevant features), the rubric used was consistent throughout. 
Students were required to demonstrate each of the assessment criteria twice to earn full grades 
and had four opportunities to do so. Given that one of the purposes of this study was to provide 
the opportunity for students to use feedback on subsequent work it should be noted that no letter 
or numeric grade was returned with the feedback sheets. Rather, for each problem, students 
received indications of which of the criteria associated with various course learning outcomes 
they had successfully demonstrated. In this way, students could focus on the feedback and not 
the grade. The potential percentage contribution to the overall course grade for each criterion 
demonstrated at least twice in four attempts was provided to students at the start of the course. 
The course grade was calculated only at the end of the course when students had exhausted all 
four attempts to demonstrate the criteria. The course and the underlying pedagogical approaches 
are described in more detail in Kuruganti and Zundel (2004). 
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Table 1 


Description of Mathematical Problems 



Problem 1 

Problem 2 

Problem 3 

Problem 4 

Factors 

Describing 

Individual 

Assignments 

Personal time, 
energy or 
financial budget. 

Financial analysis 
of a simple 
investment 
project by a small 
firm. 

Sustainability 

analysis. 

Rewrite of one of 
problems 1-3. 
Done as take- 
home 

examination. 

Information 
used to solve 
problem 

Available to them 
directly from their 
personal or family 
records. 

Most provided to 
them by 
instructor, 
selected items to 
be researched by 
groups. 

Researched by 

students 

independently. 

Depends on 
which problem 
rewritten. 

Individual or 
Group 

Individual 

Group problem 
definition, 
individual 
analysis and 
report. 

Group analysis, 
individual report 
and model building. 

Individual report. 

Conceptual 

Challenge 

Understanding 
basic problem 
solving model 
and simple time, 
financial or 
energy 

calculations (e.g, 
amount per time). 

Discounted cash 
flow analysis 
(time value of 
money), financial 
indicators, 
inflation, interest. 

Social, 

environmental and 
economic 
sustainability, 
concept of a 
quantitative 
indicator, working 
with multiple 
indicators. 

No new concepts 
introduced here. 

Who Defines 
Topic Area 
of Problem? 

Instructor 

Instructor 

Student with 
approval of 
instructor. 

Depends on 
which problem 
rewritten. 

Who Defines 
the 

Indicators 
Used to 
Assess 
Solutions to 
the Problem? 

Instructor 

Instructor 

Students with help 
from instructor. 

Depends on 
which problem 
rewritten. 

Duration 

5 weeks 

4 weeks 

4 weeks 

2 weeks 

Class 

time was also used 

[ to introduce key 

components of the problem solving, 


mathematical modelling, and economics theory content of the course. Students were provided 
opportunities to practice sub-components of the abilities being learned and assessed in the four 
class problems. 

Students received multi-channel feedback as follows. First, each student received 
individualized written feedback from the course instructors based on the rubric provided with the 
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assignment. The feedback given to students after each assignment did not contain a letter or 
percentage grade. It included only a judgment of whether the attempt at demonstrating each 
criterion was successful and advice about how to improve on or repeat their performance (when 
successful). Second, students received individual feedback on the mathematical modelling 
performance from undergraduate teaching assistants who were primarily responsible for the 
assessment of spreadsheet models developed by the students. These teaching assistants were 
trained by the instructor to give feedback and themselves received instructor feedback about their 
feedback to students. Third, verbal feedback was provided to the whole class during a class 
lecture about common issues with their papers. The course instructors also provided weekly 
opportunities for individual or small group feedback during office hours. 

Method 

This qualitative study was carried out using two cohorts of students enrolled in a 
leadership program at the University of New Brunswick in Eastern Canada. Of the 49 potential 
participants, 24 were in their first year of university and were completing the course at the time 
of the study and 25 were second year students who had completed the course the previous year. 
All 49 students were contacted via email and told about the study and were invited to participate. 
From the 14 students who responded positively, we purposefully selected (Patton, 2002) 
participants that represented the cohorts of students enrolled in the program (both genders and 
both high school entrants and mature students). The participant population selected consisted 
initially of nine students, however one second-year interview was lost due to problems with the 
voice recording equipment. Thus the final sample included five first-year students (three males, 
two females) and three, second year students (one male, two females). Of the eight participants, 
six began university studies immediately following high school graduation, and two were mature 
students with some prior post-secondary education. 

This research was reviewed by the UNB Research Ethics board. The interviewer 
reviewed the purpose of the study/interview with each participant and carried out the informed 
consent briefings ahead of each interview session. Following this initial informed consent 
discussion, the participant filled out the informed consent forms and gave them to the 
interviewer. 

An independent experienced qualitative research interviewer with graduate-level research 
experience conducted all interviews. Prior to meeting with the participants, the researchers and 
interviewer developed an interview guide, the purpose of which was to provide some structure to 
the interview. As stated, the interview questions were meant to guide the discussion and the 
interviewer took the opportunity to ask probing questions where she felt they were warranted. 
The initial questions were intended to make the participant feel at ease and for the participant and 
interviewer to get comfortable with each other. For example, the interviewer first asked where 
the participants were from and then asked them to talk about how they heard about the program 
and why they decided to enrol in the program. Eventually the interview asked more specific 
questions, such as, “Describe your progression in mathematical modelling and structured 
problem solving in the math/econ course,” and “What activities in your math/econ class are 
(were) most helpful to your improvement in problem solving and mathematical modelling?” All 
interviews were conducted in the College where students met for a number of classes. It was felt 
this was a familiar and comfortable space for students. Interviews lasted between one and one- 
half hours each. At the point when all eight interviews had been conducted, the interviewer 
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emailed the researchers stating she felt a saturation point had been reached. As a result, no 
supplementary interviews were conducted. 

Data Analysis 

All interview transcripts were audio-taped and transcribed. The constant comparative 
method of data analysis was employed (Glaser & Strauss, 1967). This method is used by 
researchers “who are not seeking to build substantive theory” (Merriam 1998, p. 159). Using this 
method, data analysis in the present study consisted of inductively derived category construction 
directly from the data and deductive categories based on constructs found in the literature. More 
specifically, the researchers conducted within case analysis followed by cross-case analysis. The 
process began with a line-by-line reading of a transcript in order to code the data. Here the 
researchers began identifying and naming segments of the data. Following the initial coding 
process, the researchers categorized the data segments by comparing and contrasting the various 
segments. The purpose here was to identify themes and patterns in the data. This process was 
repeated to (dis)confirm and add to previous thoughts and notes. Repeated readings occurred 
until no new codes and categories were added. This process was completed independently by 
both researchers. The researchers then met to discuss and compare their initial work. Through 
extensive discussion a master list of concepts derived from both researchers’ notes was created. 
This process was repeated with a second transcript; the purpose here was to look for similar 
concepts and categories and add new ones as they arose. The work from both interviews was 
compared and the work was merged into a master list of concepts and themes. Following this, the 
researchers used the master list to code a portion of another transcript to check for reliability of 
the coding. The inter-rater reliability was over 80%. 

Data Trustworthiness 

A number of steps were taken to ensure the credibility and trustworthiness (Patton, 
2002) of the data. First, multiple cases were investigated. Second, transcripts were shared with 
participants to ensure that they represented participants’ thoughts. Participants had the 
opportunity to clarify or delete material. Third, the two authors conducted reliability checks to 
ensure the accuracy of coding. 


Results 

The results describe how participants experienced a course that intentionally incorporated 
the following three feedback practices: (a) consistent and transparent grading practices that were 
known prior to the assessment of work (via the use of a rubric); (b) the opportunity to use the 
feedback received on previous work in subsequent work; and (c) the provision of various types 
of feedback (written, verbal, individual, and group). Where appropriate, the intentional 
pedagogical practices employed will be used as the overall category title with sub-themes 
relating to each category discussed within each category. The two categories presented below 
relate to consistent and transparent grading practices and provision of multi-channel feedback. 
Each of these is divided into themes that group participant observations. The participants’ views 
are presented in each category/theme and the particular quotations have been selected because 
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they typify common perspectives found within each theme. To protect the identity of the 
participants, all names used are pseudonyms. 

Consistent and Transparent Grading Practices 

A rubric that described the abilities students were expected to demonstrate was provided 
in conjunction with the first assignment. The same rubric was used to grade each assignment. 
Written feedback was provided to students on their submitted assignments through the use of the 
rubric. Students found it extremely useful that the feedback indicated what they had done 
correctly, and it also provided guidance for where to improve their performance on the following 
assignments. Three themes emerged within this category. 

Useful to guide future work. Participants repeatedly commented on the usefulness of the 
rubric. In particular, participants described how they used the rubric as a guide to assist them 
with their assignments. Joan stated, “... when I noticed that [there were] certain checkmarks 
[that] I ... failed to get the first time and the second time around I was like ‘Okay this is what I 
need to concentrate on.’ Paul stated, “ ... if I missed a checkmark on the first problem then on the 
second problem I ... made sure that ... I made a conscious effort to ... incorporate [it]....” When 
discussing how his papers improved from the first to second problem, John said, “I looked at my 
last paper and saw ... what I did and then how they said I could improve and then based the 
structure off of that.” This must have been an important consideration to John as he referred to it 
again later in his interview when he said, “It just, it showed me what I did wrong and how I 
could fix it for the next problem and I understood what I did.” As another example, Lisa offered 
the following comment: 

I guess just because ... we followed that marking grid every time we had an 
assignment; we were like forced to do it. ... So ... if you [had] just been [told] ... 

“these are the concepts and you should use them while doing your [assignment]” I 
don’t know if I really would have followed them as closely, so having to do that ... 
engrains it on your mind so that by the end of it you were just automatically using it 
and then you could write your paper and then look back at the grid and make sure 
you had everything, not use the grid to write the paper like I would have for the 
first [problem]. 

The opportunity for students to use the feedback acquired from previous problems 
to guide subsequent action was dependent on students knowing that subsequent problems 
would be assessed using the same rubric as previous ones and would provide further 
opportunities to demonstrate the same abilities. 

Provided focus for follow-up conversations. When discussing the use of the rubric 
students repeatedly talked about how the written feedback helped them focus their attention on 
specific questions or issues they needed clarified. That is, students read the written feedback and 
then formulated questions for which they sought answers from the instructors during office 
hours. Rather than approaching the professor with broad questions, the students went seeking 
answers to specific questions. As Joan explained, 

Like you’d get [written] feedback so if you didn’t or you kind of [demonstrated 
concepts] Pierre would write, “okay, I kind of got the feeling ... [that you partially 


http://ir.lib.uwo.ca/cjsotl_rcacea/vol3/iss 1/3 
DOI: http://dx.doi.org/10.5206/cjsotl-rcacea.2012.L3 



Randall and Zundel: Assessment as a Learning Tool: Student Perspectives 


understood] but perhaps a little bit more elaboration on what ...you mean[t] by 
this situation.” Which is great so you could look at your feedback and look where 
you needed to improve and then you could make up your questions or if you had 
any questions you could approach Pierre and [he would] point out where you 
....[got] things wrong and [you could] ask questions. 

Can’t ignore. The third theme to emerge was that students found this type of feedback 
could not be ignored. In response to the interviewer’s prompt about the use of the feedback 
provided on one his papers, Robert, whose answer was representative of all the students when 
asked a similar questions, stated, “For sure, you’d have to, to do well in the course.” 

Deeper understanding and use of content. Five of eight participants also reported that 
the use of the same rubric for multiple similar problems provided opportunities to understand 
both the generic abilities (e.g., problem-solving) and the “content” of the problems (e.g., 
financial analysis concepts). Lisa explains, 

I guess I just ... came to better understand the terms [in] the first problem; I 
thought I knew what I was talking about and I really didn’t and then I went to 
Pierre and got... more information and then [in] the lectures more information was 
given and I guess just throughout the term I came [to] better and better understand 
what the different terms and concepts entailed and like what I was expected to 
write about and... notice in the problem. So ... the understanding of what the terms 
meant and how to go about explaining them and being able to use math models to 
solve the problems [improved] because I struggled with that a lot on the first one 
but then once I had that as my example to refer back to I found it easier to figure 
out how to do the other two (Lisa). 

The Provision of Multi-Channel Feedback 

In addition to the rubric, two other forms of feedback were also given to students 
throughout the course. First, the professor provided general feedback to the entire class about 
issues that were of wide-spread concern when assignments were returned to students. Second, 
students had the opportunity to seek further individual verbal feedback by way of meetings with 
instructors during instructors’ set office hours. Related themes that came out during participant 
interviews were.... 

General verbal feedback provided in class. Following the grading of assignments, the 
instructors would use lecture time to review and discuss with the class the common issues that 
students appeared to do well with and struggle with. These discussions appeared to help students 
locate “where they were” in the course. For example, Paul said, “Well, after we got the 
assignments back Pierre and Mike always [went] over them and ... [pointed out] strong points, 
weak points, things everyone should consider for next time.” 

Interview participants found it useful to listen to other students’ questions during the 
lectures where general comments were made about trends in student work on a particular 
assignment since it helped them clarify their own thinking. For example, Carla noted, 

What would sometime happen that was really helpful was, the class would have a 
series of questions. That was really helpful because when somebody asked a 
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question I liked hearing the answer to their question and that was worthwhile. I 
think they were probably the most useful. 

Personalized verbal feedback. Students had the opportunity to seek feedback through 
regularly scheduled office hours. When asked about the types of things students discussed during 
this time, they repeatedly referred to questions/topics that were directly related to the previous 
written feedback they had received on their assignments. The students sought personalized 
feedback. That is, they wanted information specifically related to what they did wrong and how 
they could have improved. The individualized nature of requested feedback was quite strong as 
demonstrated in the following two quotes: 

I don’t know, I guess just because he could tell me ...specifically what I had [to 
do.] [I’d ask] “well what else do I need?” kind of thing or what am I lacking? 
Because [with] the general explanations [provided]... you think you have it and 
you get your assignment back and you didn’t get a checkmark for it and so just to 
know specifically what you left out or what you need to elaborate on more ... was 
more helpful than just the general gist of what you need to do (Lisa). 

Talking to Pierre. Because his approach in terms of asking you questions would 
be like, “Okay let’s look at this first, let’s look at this now.” Like that approach of 
looking at one thing at a time kind of helped me out in terms of “okay, this is what 
I need to do when I work it out” (Joan). 

Discussion 

This research set out to explore how students experienced a course that built in evidence- 
based practices that help students make effective use of feedback in their learning. The results 
demonstrate that students felt the positive effect of multi-channel feedback. In addition, the 
students’ learning improved as a result of criterion-referenced grading (i.e., instructor comments 
on graded papers with the accompanying rubric, general comments provided in class with the 
opportunity to ask questions, and verbal comments during office hours). The eight students in 
this study all commented on the fact that they took advantage of more than one feedback 
practice. Further, their motivation to obtain this feedback was largely related to the opportunity 
and need to use the feedback in subsequent assignments. 

One of the things we believe may be happening here that warrants further investigation is 
that a synergy developed between the use of a consistent rubric, the multiple channels through 
which students were provided feedback, and repeated opportunities to apply developing skills 
and knowledge. This is consistent with the findings of Biggs (1999) who reported synergistic 
effects of combining multi-channel feedback referenced to consistent assessment rubrics on 
student learning. In this study, we also found that patterns in criterion-referenced written 
feedback over several assignments helped focus students’ questions during face-to-face contact 
with instructors. Students attended to feedback because they were confident that it would be 
useful to them in subsequent attempts (and possibly because they had no grades on which to 
focus). We have shown that that the students recognized an increase in their competency over the 
repeated attempts. This is consistent with the work of Alvemo College Faculty (1994) who 
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characterized assessment as learning when explicit, behavioural criteria are used to provide 
feedback to students on their performance. 

When the criteria used in the assessment rubric were derived from descriptions or 
decompositions of the complex abilities we hoped students would learn in the course, and when 
the repeated problems given to students provided opportunities to exercise them, a situation was 
created in which assessment became a key means of having students close the gap between their 
initial ability level and that desired. Without this explicitness in expectations, students in this 
study were hard pressed to know where to concentrate their efforts and how to leam from early 
failures. These findings are consistent with the framework proposed by Gibbs and Simpson 
(2003) who describe conditions in which assessment supports student learning. The authors 
identify the use of clear criteria that relate to the purpose of the assignment, coupled with 
prompt, detailed feedback using these criteria and opportunities for students to use it in 
subsequent assignments as key conditions for effective feedback. This is also consistent with the 
findings of Ellery (2008) and Higgins et al. (2002) who indicate that students can use formative 
feedback to learn from their mistakes. 

Being provided the assessment rubric ahead of time was something new for students and 
needed substantial explanation during class time. Initially, they were not sure what to do with it 
or how it would be used. However, after the first paper had been returned and the rubric used, 
students found the rubric provided clarity and direction as they quickly understood where they 
needed to direct future efforts in order to demonstrate understanding of the concepts. It appeared 
that this feed forward information (Higgins et al., 2001) provided direction for students in a 
course that many students struggled with; the direction helped counter the effects of the difficult 
material. 

We believe the rubric helped in another way: the students enrolled in the program of 
study were high achieving students (only 30 admitted per year to the degree program). These 
were students who had little experience struggling with content or having to ask for help. The 
rubric and the accompanying comments allowed some students to overcome the fear of asking 
for the help they needed from instructors. 

Participants in this study clearly favoured the specific nature of feedback provided after 
each attempt, which is in keeping with previous research. However, the participants also found 
that the general feedback provided to the class about the assignments overall to be effective. This 
is likely due to the fact that the general feedback provided was more in line with Knight and 
Yorke’s (2003) definition of general feedback, that being that the feedback was more structural 
in nature and directly related to expectations that were met and needed to be met for the next 
assignment. 

Our results align well with four major prior research findings: first, Biggs’ (1999) 
contention that aligned instruction (objectives, teaching, and assessment) is more likely to lead to 
deeper student learning; second, Paulaos and Mahony’s (2007) finding on the importance of 
having criteria to complete the assignment and the assessment matching the criteria that had been 
previously provided; third, Orsmond, Merry, & Reiling’s (2005) recommendation that “student 
progression and student feedback need to be more closely linked. Students want to have 
feedback regarding their coursework put into the wider dimension of their overall progression” 
(p. 382); finally, Price, Handley, Millar, & O’Donavan’s (2010) finding that students want 
dialogue with instructors about the feedback provided. 

Previous research has suggested that students use feedback in various ways; some 
students adhere closely to comments while others store them for future reference (Higgins et al., 
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2002). This finding led researchers to suggest an exploration of how students use feedback as an 
area of future investigation (Higgins et al., 2002). Although this study did not set out to explore 
this question, our interview data provided some interesting insights relative to it. Students 
commented quite clearly that they had to use the feedback if they were to be successful on 
successive attempts. Because there was no grade to consider, students were more apt to closely 
examine and seek clarification of feedback on their work to guide and improve future efforts. 

It should be noted that the multi-channel feedback and individualized criterion-referenced 
assessment described in this paper are resource intensive. That is, this course used a substantial 
amount of instructor (20-30 minutes per paper for four papers per student) and teaching assistant 
effort (six hours per week for ten weeks of the semester) for a modest number of students. In 
situations with fewer resources or greater numbers of students, the time required to provide 
individualized feedback would quickly become onerous. As a result, the choice of course 
learning objectives, the complexity and scale of assignments and assessment rubrics and the total 
number of assignments will need to be congruent with the instructional resources available. 

Conclusion and Implications 

We set out to explore how assessment practices and the students’ perceptions of them 
affected their learning. During the interviews all eight participants talked about the practices and 
described how they felt each contributed to their learning. Although each practice was referred to 
individually, as well as in combination, by students, it became apparent to us that it was the 
combined effect of the practices that really impacted them. For example, students used the 
grading rubric as a guide to write their papers. That same rubric was used to provide detailed 
feedback to students on the extent to which they met the criteria. Knowing they would have 
another opportunity to demonstrate criteria not attained on one attempt, the students sought 
further clarification of how to overcome weaknesses through personalized one-on-one 
conversations with the professor. 

The combined use of the pedagogical practices impacted students in three ways. First, 
they were motivated by the combined use of these practices to invest effort and attention in 
learning in areas where they needed growth, in spite of the very challenging nature of the subject 
matter. This motivation stemmed in part from the feedback they received, which identified both 
their strengths and areas for improvement, and from the further opportunities to demonstrate 
their growing skills that the subsequent problems provided. Second, the practices allowed 
students opportunities to focus their efforts on areas requiring growth, thereby helping them 
make good use of limited learning time. Finally, this focus by students on particular learning 
needs also led students to seek out (and the instructors to provide) specific instructional 
opportunities targeted to their needs, such as explanations of challenging concepts during 
instructor office hours. 

In this paper we have focused on assignments and feedback mechanisms, but there are 
other structures that may play an important role in student learning. Based on our transcripts, two 
key areas for further investigation include: (a) the role of informal support networks (e.g., 
student peer groups) in maintaining student motivation and providing learning opportunities; and 
(b) the impact of not providing students actual grades on assignments on the level of attention 
paid to formative feedback. 
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Appendix 

RCLP 1052: Mathematical and economic approaches to problem-solving 

Problem 1 Grading Sheet 


Student: 


Outcome 

Criterion Description 

Adequate 

Comments 

Math Modelling 
(MM) 

1. Calculate personal financial 
situation correctly 



2. Make appropriate inferences 
about financial situation. 



3. Spreadsheet model can be 
effectively used to carry out 
quantitative analysis on budget 



4. Justified choice of variables for 
sensitivity analysis (SA) 



5. Set reasonable range of values 
for SA 



6. Made appropriate inferences 
from SA results 



Structured 

Problem-solving 

(PS) 

1. Rephrase or describe problem 
accurately 



2. Identify constraints, objectives 
and resources 



3. Define key terms subject to 
ambiguity 



4. Define at least 3 several 
possible strategies to improve 
financial situation 



5. Predict performance of each 
strategy in terms of objectives and 
constraints 



6. Select optimal strategy and 
justify choice 



7. Identify assumptions and 
predict their effects on financial 
situation. 



8. Observe own work accurately. 



9. Analyse work with PS, MM 
and ML criteria 



10. Judge quality of work 
accurately 



11. Implement plan from last self- 
assessment. 
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Working with 
quantitative 
information 
(multi-literacy - 
ML) 

1. Use appropriate format to 
present information (table, type of 
graph or figure) 



2. Labelling and caption is clear 
and effective 



3. Appropriate use of quantitative 
information in report to support 
description and conclusions. 




Other comments: 
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